Random Variables & Probability distributions

Discrete variables

For a discrete variable X that can take values from $a_{1}, a_{2}, . . .$ , there is a probability function for X:

p_{i} = P (X = a_{i}) (i = 1, 2, . . .)

There are two major types of distributions for discrete variables (based on $p_{i}$ ): Binominal distribution, and Poisson distribution.

X \sim B (n, p)

p_{i} = b (i; n, p) = (\binom{n}{i}) p^{i} (1 - p)^{n - i}

example: observe $X$ times of heads when flipping coins for N times. Then X=1, 2, ..., N follows this distribution.

\bar{X} = n p

V a r (X) = n p (1 - p)

Bernoulli distribution

The Bernoulli distribution is a special case of the binomial distribution with $n = 1$ .

X \sim P (λ)

P (X = i) = e^{- λ} \frac{λ^{i}}{i!}

It always works when X represents the number of events that happen within a temporal or spatial domain.

Poisson is an extreme case of binominal distribution:

for $X \sim B (n, p)$ , when n is large & p is small & np = \lambda is not too large, then $X \sim P (λ)$ .

For continuous variables, probability density functions are more useful than cumulative distribution functions.

For a continuous variable X with its CDF F(x), there is a probability density function of x:

f (x) = F^{'} (x)

The probability density function has 3 features:

f (x) \geq 0

\int_{- \infty}^{\infty} f (x) d x = 1

P (a \leq X \leq b) = F (b) - F (a) = \int_{a}^{b} f (x) d x

Important

The PDF's analog for discrete variables is probability mass function (PMF). But, they are not same!!!
PDF is not Probability! It only means how much probability is concentrated per unit length (d𝒙) near 𝒙, or how dense the probability is near 𝒙.
For discrete random variables, we look up the value of a PMF at a single point to find its probability P(𝐗=𝒙).
For continuous random variables, we take an integral of a PDF over a certain interval to find its probability that X will fall in that interval.
Thus, PDF can be greater than 1, such as in an exponential distribution:

There are several major types of distributions for continuous variables (based on PDF):

X \sim N (μ, σ^{2})

f (x) = \frac{1}{\sqrt{2 π} σ} e^{- \frac{(x - μ)^{2}}{2 σ^{2}}}

X \sim e x p (λ)

f (x) = λ e^{- λ x} (x > 0)

The exponential distribution is a special case of the Weibull distribution with \alpha = 1.

PDF:

f (x) = λ α x^{α - 1} e^{- λ x^{α}} (x > 0)

CDF:

F (x) = 1 - e^{λ x^{α}}

f (x) = \frac{1}{b - a} (a \leq x \leq b)